Microsoft® Exchange is a messaging and collaboration software system that provides a variety of applications for group interaction using networked computer systems. Specifically, Microsoft Exchange (by Microsoft Corp. of Redmond, Wash.) provides support for a database server to support remotely connected clients over various networks. An Exchange server can run on a variety of operating systems including, for example, the Microsoft Windows NT® or Microsoft Windows® 2000 operating systems.
The Microsoft Exchange 2000 database server permits up to 20 databases to be utilized, divided into four storage groups of up to five databases each. In a typical configuration, Microsoft Exchange stores data associated with a given database in two files. In the particular example of Microsoft Exchange 2000, these two files are a properties data file (designated by an .edb file extension) and a streaming data file (designated by an .stm file extension). However, it is contemplated that other versions support different database structures. The default storage locations for these databases are on a disk locally connected to the computer on which the Exchange software is running.
In the exemplary Exchange 2000 server configuration, the .edb file is a conventional Exchange database file. The .stm file is a streaming data file that contains raw content that has been received via, for example, the Internet and is stored in its native format. Pointers are created by the Exchange server within the .edb file to point to various messages or data within the .stm file.
FIG. 1 shows an exemplary flow chart of data passing through an Exchange server. In step 105, the data that is generated by a client of the Exchange server is a transaction. This may be, for example, email messages and associated attachments created by an email client, such as Microsoft Outlook. Next, in step 110, the transaction is stored in the memory of the server. The storage of the transaction in memory is often transient in nature until the message is committed to some form on nonvolatile storage. The transaction is then written to a log file in step 115. The log files typically have a pre-allocated size. In one known example, each log file is 5 megabytes (MB) in size. In an illustrative embodiment, Microsoft Exchange requires that the most recent log file be named “e0n.log” where n may be a value in the range of 0-3 representing up to four storage groups. Each preceding log file is named “e0nXXXXX.log.” The “XXXXX” is a numerical identifier associated with the particular log file. Thus, an Exchange server may have a variable number of log files at any given point-in-time, depending on how many log files have been incorporated into the database files. As a further step, the transaction that is written to the log file is also written from system memory to the database file (step 120). The writing of the transactions to database occurs in a lazy write fashion. By “lazy write” it is meant a writing process or procedure of the Exchange software that performs the write when central processing unit cycles are available.
FIG. 2 is a schematic block diagram of an exemplary Exchange server environment 200. An exemplary server 205 executing the Windows® 2000 Advanced Server operating system containing a local disk 210 is shown connected to a backup tape drive 220 and an external disk 215. The external tape drive 220 is connected via either a small computer system interface (SCSI) connection or a switching network, such as a storage area network (SAN). Similarly, the external disks 215 may be connected via a SAN or other suitable networking architectures. The Exchange server 205 may be incorporated into a Microsoft Clustering System (MSCS) environment 225. Such a clustering environment provides for redundant data program access to clients.
In known examples of Exchange servers, the Exchange software provides an application program interface (API) that is accessible by other programs executing on the server for performing backup and restore operations on the various databases. Other applications or processes executing on the server can access these APIs to perform various backup/restore operations. These API's are targeted toward the use of a tape drive as a backup storage device. Such backup operations are normally performed while the Exchange server is operating. As tape drives typically have a slower read/write time than disk drives, the backup of databases with a tape device can consume a significant amount of time. While the Exchange server continues to service requests during a backup operation, performance is degraded during the course of the backup operation. Because of this and because tape devices require significant time to perform a backup, such backups are typically performed at night (or other off-peak time), when few users are utilizing the system. Similarly, a database restore operation using a tape device consumes a substantial amount of time, also causing concurrent system degradation. When performing a backup or restore operation, the database files and any unincorporated log must be saved and/or restored. Thus as the size of the various database files increases, the time required to perform a backup/restore operation to a tape device also increases.
In a further known example, the Exchange server is adapted to have the database and log files preferably written to a local disk. However, by utilizing other software products such as, e.g. SnapManager® for Microsoft Exchange 5.5 available from Network Appliance, Inc. of Sunnyvale, Calif., the log files and databases may be written to disks that are remotely connected to the Exchange server. In one known implementation, the Exchange server is operatively interconnected with a file server and associated disk arrays, which provides file service for storage and access of the database and log files.
A file server is a computer that provides file service relating to the organization of information on storage devices, such as disks. The file server or filer includes a storage operating system that manages data access and may implement a file system to logically organize the information as a hierarchical structure of directories and files on the disks. By “file system” it is meant generally a structuring of data and metadata on a storage device, such as disks, which permits reading/writing of data on those disks. A file system also includes mechanisms for performing these operations. Each “on-disk” file may be implemented as a set of disk blocks configured to store information, such as text, whereas the directory may be implemented as a specially-formatted file in which information about other files and directories are stored. A filer may be configured to operate according to a client/server model of information delivery to thereby allow many clients to access files stored on a server, e.g., the filer. In this model, the client may comprise an application, such as a file system protocol, executing on a computer that “connects” to the filer over a computer network, such as a point-to-point link, shared local area network (LAN), wide area network (WAN), Storage Area Network (SAN) or virtual private network (VPN) implemented over a public network such as the Internet. Each client may request the services of the filer by issuing file system protocol messages (in the form of packets) to the filer over the network.
A common type of file system is a “write in-place” file system, an example of which is the conventional Berkeley fast file system. In a write in-place file system, the locations of the data structures, such as inodes and data blocks, on disk are typically fixed. An inode is a data structure used to store information, such as meta-data, about a file, whereas the data blocks are structures used to store the actual data for the file. The information contained in an inode may include, e.g., ownership of the file, access permission for the file, size of the file, file type and references to locations on disk of the data blocks for the file. The references to the locations of the file data are provided by pointers, which may further reference indirect blocks that, in turn, reference the data blocks, depending upon the quantity of data in the file. Changes to the inodes and data blocks are made “in-place” in accordance with the write in-place file system. If an update to a file extends the quantity of data for the file, an additional data block is allocated and the appropriate inode is updated to reference that data block.
Another type of file system is a write-anywhere file system that does not overwrite data on disks. If a data block on disk is retrieved (read) from disk into memory and “dirtied” with new data, the data block is stored (written) to a new location on disk to thereby optimize write performance. A write-anywhere file system may initially assume an optimal layout such that the data is substantially contiguously arranged on disks. The optimal disk layout results in efficient access operations, particularly for sequential read operations, directed to the disks. A particular example of a write-anywhere file system that is configured to operate on a filer is the Write Anywhere File Layout (WAFL™) file system also available from Network Appliance, Inc. of Sunnyvale, Calif. The WAFL™ file system is implemented within a microkernel as part of the overall protocol stack of the filer and associated disk storage. This microkernel is supplied as part of Network Appliance's Data ONTAP™ storage operating system, residing on the filer, that processes file-service requests from network-attached clients.
As used herein, the term “storage operating system” generally refers to the computer-executable code operable on a storage system that manages data access and may implement file system semantics. In this sense, Data ONTAP™ software is an example of such a storage operating system implemented as a microkernel. The storage operating system can also be implemented as an application program operating over a general-purpose operating system, such as UNIX® or Windows NT®, or as a general-purpose operating system with configurable functionality, which is configured for storage applications as described herein.
Disk storage is typically implemented as one or more storage “volumes” that comprise physical storage disks, defining an overall logical arrangement of storage space. Currently available filer implementations can serve a large number of discrete volumes (150 or more, for example). Each volume is associated with its own file system and, for purposes hereof, volume and file system shall generally be used synonymously. The disks within a volume are typically organized as one or more groups of Redundant Array of Independent (or Inexpensive) Disks (RAID). RAID implementations enhance the reliability/integrity of data storage through the redundant writing of data “stripes” across a given number of physical disks in the RAID group, and the appropriate caching of parity information with respect to the striped data. In the example of a WAFL-based file system, a RAID 4 implementation is advantageously employed. This implementation specifically entails the striping of data across a group of disks, and separate parity caching within a selected disk of the RAID group. As described herein, a volume typically comprises at least one data disk and one associated parity disk (or possibly data/parity) partitions in a single disk) arranged according to a RAID 4, or equivalent high-reliability, implementation.
A file server, as described above may be interconnected by a network to an Exchange or other database server to provide file service operations. In the example of an Exchange database server, the various database files can be stored on a set of disks interconnected with a file server through the use of such software programs as the above-described SnapManager software. As noted, such file servers typically utilize a tape device for backup/restore operations. A substantial amount of time and system resources are required to perform a backup operation to a tape device. Consequently, many system administrators do not frequently perform backup operations, to avoid the concomitant system performance degradation.
When a database or other file becomes corrupted, the administrator may desire to restore the file using a previously generated backup image of the file. Using conventional tape-based restore operations, an administrator would take the database server offline and then copy the restored database files to the active file system. This copying could take a substantial amount of time (on the order of hours) depending upon the size of the restored database file. Depending upon the methodology used in generating a backup of the file, a hard or soft recovery procedure could be utilized.
There are known methods for performing a soft recovery of an Exchange database. A soft recovery is employed when an Exchanger server is taken offline and then reinitialized. Upon restart, the Exchange server replaces transaction logs to bring the database to a current and consistent state. One example of a soft recovery is described in U.S. patent application Ser. No. 10/090,963, entitled SYSTEM AND METHOD FOR CREATING A POINT-IN-TIME RESTORATION OF A DATABASE FILE, by Dennis E. Chapman, the contents of which are hereby incorporated by reference.
However, soft recovery is not an appropriate method to restore databases being served by a database server that is handling multiple databases. Significant advantages in reliability, scalability and flexibility are realized by having a single database server manage multiple databases. Specifically, the other databases being managed by a database server will be able to remain online when one database needs to be recovered. In the example of Exchange 2000, which supports multiple storage groups and multiple databases per storage group, a hard recovery is the preferred recovery method.
A hard recovery is the act of restoring the server from an initial or empty state. The database is restored from a backup and all transaction logs are replayed. During a hard recovery, the database server reconstructs the database from a backup data file. However, while the online backup was being created, often there will be transactions that are received by the server between the time the database is written to the backup location and the time that the backup finishes. These additional transactions are stored in patch files. When executing a hard recovery, these patch files are also added to the database so that the recovery is complete. Patch files are no longer utilized with Exchange 2000 servers running Service Pack 2 (SP2) or higher. However, many installations are in service that are not utilizing SP2.
A noted disadvantage of hard recovery operations is that they are typically performed from a backup stored on a tape device. By utilizing a tape device for backup operations, system performance is degraded and the time required for the backup operation is substantially longer than for soft recovery operations utilizing a snapshotting file system. Thus, it is an object of the present invention to provide a system and method for performing a backup and restore operation utilizing a hard recovery procedure for a database.